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Abstract: A survey was designed incorporating questions on 28 attributes (compiled through a 
literature review) and considered to be quality features in online academic courses in higher 
education. This study sought to investigate the ongoing practice of instructional designers and 
instructors in the United States with respect to their incorporation of these quality best practices 
into their design process and course content. Although most respondents indicated they included 
the majority of the quality attributes in courses they designed or taught, a significant number 
rarely or never included some of the generally accepted features shown to result in positive learner 
outcomes. 

Keywords: online learning, quality, higher education, instructional design 

Resume : Nous avons mene une enquete dont les questions portaient sur les 28 criteres de qualite 
(identifies dans le cadre d'une revue de litterature) des cours en ligne dans l'enseignement 
superieur. Cette etude cherchait a faire ressortir dans quelle mesure les bonnes pratiques definies 
concernant la qualite sont mises en ceuvre par les concepteurs pedagogiques et les enseignants, aux 
Etats-Unis, dans leur processus de conception des cours et le contenu de ces derniers. Bien que les 
repondants indiquent, pour la plupart d'entre eux, qu'ils integrent la majorite des criteres de 
qualite dans les cours qu'ils congoivent ou enseignent, un nombre significatif d'entre eux n'integre 
que rarement voire jamais certains de ces criteres communement consideres comme ayant un 
impact positif sur la reussite des etudiants. 

Mots-cles : apprentissage en ligne, qualite, enseignement superieur, conception pedagogique 

Introduction 

The number of postsecondary students in the United States enrolled in at least one online 
course for academic credit continues to rise. In 2013, an analysis of the most recent data from 
the Integrated Postsecondary Data System (IPEDS), showed more than 5.5 million students 
were enrolled in at least one fully online course in the U.S. (Allen & Seaman, 2015; Poulin & 

Straut, 2015). In addition, of all degree granting institutions of higher learning in the U.S., 
about 70% now have some form of online distance offering, and academic leaders (77% in 2013 
and 74% 2014) have begun to understand that online learner outcomes can be the same or 
superior to face-to-face class instruction (Allen & Seaman, 2015). Yet, while a strong majority 
(70%) of academic leaders report that online learning is critical to their long-term strategies, 
only about a third (28%) of Chief Academic Officers say their faculty accept the value and 
legitimacy of online education (Allen & Seaman, 2015). Online course offerings are being 
released at a faster rate than current faculty are being trained to deliver quality education in a 
digital age. Faculty are challenged to keep up with the plethora of digital applications and 
approaches to course delivery now available, how to incorporate technology into the learning 
environment, and how to ensure their choices will result in the highest quality learning 
experience for the student. A recent survey by the Online Learning Consortium (OLC) 
showed that a high percentage of faculty had not adopted new learning techniques in their 
courses, were not familiar with the technologies or determined the technologies weren't 



This work is licensed under a Creative Commons Attribution ShareAlike 4.0 International License. 






relevant (Allen & Seaman, 2015). This research examined the attributes of quality, including 
defining quality in an online context, and theories that inform instructional design practice and 
survey methodologies that underpinned the framework for this study (Cresswell, 2013; Reiser, 
2001; Dick, Carey and Carey, 2004; Reigeluth, 1999). 

As part of a series of independent study credits offered to students in the Masters of 
Educational Technology (MET) at the University of British Columbia in Canada and conducted 
by a graduate student investigator, who is also first author, between May 2015 and December 
2015, this study examined various attributes of what constitutes quality measures in an online 
learning environment. The MET graduate program is delivered online and the student 
investigator, living in the United States, elected to include only research participants in the 
US. The question now is not whether we should integrate technology into learning but how to 
use it to personalize learning, create more accessible and equitable digital environments, and 
to do this by using the flexibility and creativity that technology affords. The emphasis in the 
literature now is on ensuring that instructors are informed and trained in how to create quality 
online and blended learning spaces (U.S. Department of Education, 2010 & 2016). 

The purpose of this study was to examine online asynchronous academic credit courses in 
higher education with a view to the presence of quality attributes that are critical to their 
design and delivery. The research questions were developed as a result of the student 
investigator's work with faculty in the development of online graduate level courses at 
universities in the United States. There were two main questions: What components in an 
online course in higher education are defined as quality in the literature? and. Are instructors 
and instructional designers incorporating quality features into online courses? 

The Literature 

The determination of what constitutes measures of quality in online higher education varies 
widely in the literature. Quality is often measured as student satisfaction with an online course 
and it is understood that quality is considered low if the rate of attrition is high (Grace, 
Weaven, Bodey, Ross, & Weaven, 2012; Patterson, 2007). Quality can also be attained through 
the process of course design (a set of standards created by either the instructor, the 
instructional designer, or externally). Although Quality Matters (QM), a peer-review process 
of only faculty instructors, uses a standardized rubric to measure and certify the quality of 
online courses, the research of Monroe (2011) concluded that there was a difference between 
how instructional designers and faculty reviewers rated the quality of a course. 

Pina and Bohn (2014) focused on two important points: quality assessment rubrics that assume 
the individual who created the course is also the one teaching the course and that these same 
rubrics only focus on course design and not on the role of instructor delivery. Additionally, 
they point out that faculty are often evaluated by students who take courses that the faculty 
member may not have designed. The concept of quality can be nebulous at best (Mitchell, 

2010), or subjective and found, like beauty, in the eyes of the beholder (Maringe & Sing, 2014). 
Bates (2015) offered a definition of quality that was most appropriate as a benchmark for this 
research: "...teaching methods that successfully help learners develop the knowledge and 
skills they will require in a digital age" (Chapter 11, p. 1). 

Prior to determining what features constitute quality in online learning, it is important to keep 
in mind principles of quality teaching in general. Chickering and Gamson (1987) outlined 
seven principles of teaching undergraduates based on their research on good teaching 
practices. Chickering and Ehrmann (1996) later recognized technology (specifically computers) 
as a resource for learning that was being incorporated into undergraduate education. Merrill 
(2001) focused on the elements of teaching that include problem-based learning, content 
having relevancy to and engagement in solving real-world problems, and the demonstration of 
new knowledge - in other words, active learning as a strong element of quality teaching. 
Graham, Cagiltay, Lim, Craner, and Duffy (2001) expanded on the basic principles of 
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Chickering and Ehrmann (1996) to assess the quality of online courses and included 
technologies aligned with those same principles of good teaching. Fundamentally good 
practice encourages: student-faculty contact; cooperation among students, active learning, 
prompt feedback, and time on task; as well as high expectations and respect for diverse talent 
and ways of learning. (See Appendix l)Puzziferro and Shelton (2009) revisited Chickering and 
Ehrmann (1996), and although they affirmed the basic principles they also noted that the 
landscape of higher education was constantly evolving, as were our values. This work 
(Puzziferro & Shelton, 2009) illustrated how learning is no longer a product that is delivered 
but one that is experienced by the learner. Therefore, the role of faculty development (and 
professional instructional design) is increasingly more important now in guiding faculty on 
how to create learning spaces, specifically with the use of technology online and how to help 
instructors adapt to the new roles expected of them. 

Creating an online course is a complex and iterative process, one in which it is difficult to 
ascribe any one component as being the major contributor to its success or quality level. 
Puzziferro and Shelton (2008) support having a "common framework for consistency, design, 
pedagogy and content" (p. 119) and Simpson and Benson (2013) in support of this, continue to 
show that student satisfaction levels increase after an online course went through a systematic 
review for quality. 

Rubrics have been used for many years as a judge of quality - quality of the program, quality 
of the course and, of course, quality of the work of the individual student within a course. A 
large number of studies have been done using the Quality Matters rubric, which was designed 
in 2006 - 2007 to measure quality in online course design (Newhouse, Buckley, Grant, & Idzik, 
2013; Parscal & Riemer, 2010; Pina & Bohn, 2014; Roehrs, Wang, & Kendrick, 2013). Later in the 
decade, Shelton (2010), with the support of the Online Learning Consortium, built an 
instrument that provided a 360-degree assessment for institutions to measure the quality of 
their online programs. Shelton emphasized the importance of determining and assuring 
quality in online programs and further stated that institutions to date have not been attentive 
enough to implementing methods to audit and measure the quality of programs, or have had 
the approach that institutions "recognize quality because—we are researchers, we maintain 
accreditation, we have multiple resources at our disposal, and we are selective in our 
admissions process" (Shelton, 2011, p. 7). 

Universities are under increasing scrutiny by both governments and consumer-learners 
regarding the quality of their educational offerings. (Grace, 2013; US Department of Education, 

2010) . In the United States, academic programs in higher education are accredited through a 
peer-review process and the submission of self-studies examining compliance with the 
standards set by one of five regional accrediting bodies. The Middle States Commission for 
Higher Education, for example, has adopted the Interregional Guidelines for the Evaluation of 
Distance Education (Online Learning) which has nine hallmarks of quality that promote best 
practices in the area of program design and institutional governance and oversight (MSCHE, 

2011 ) . 

Pina and Bohn (2014) caution not to pay too much attention to course design as the only 
measure of quality but to focus attention on the actions of the instructor during course 
delivery. Jaggars and Xu (2016) concluded that the connections that learners made with the 
instructors in the course contributed to positive learner outcomes. Examples include instructor 
presence in discussions and synchronous chats or some other demonstration of caring. By 
using faculty development and mentoring as hallmarks one is more likely to produce a quality 
online course (Barczyk, Buckenmeyer, Feldman, & Hixon, 2011; Maringe & Sing, 2014). 

The availability of training, development, and resources for faculty has been known to 
improve the quality of an online course (Herman, 2012). These resources can include 
mentoring, consultant services, reading material, workshops, and group discussions. 

However, the majority of faculty teaching online were dissatisfied with the level of 
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institutional support, specifically faculty development offered by their institutions (Herman, 
2012), as many faculty had not previously taught online. Therefore, the provision of budgetary 
allocations, via institutional support, is imperative to online initiatives. Without these, the 
quality of the courses may be diminished. 

Another primary method of assessing quality is through student perception and satisfaction 
surveys (Anderson, Tredway, & Calice, 2015; Harrison, Gemmell, & Reed, 2014; Hill, 2014; 
Marmon, Vanscoder, & Gordesky, 2014). Ciobanu and Ostafe (2014) explored the connection 
between student satisfaction and learning. Student satisfaction surveys can serve as a 
formative assessment for course designers as student preferences can change over time; what 
was a desirable feature a decade ago is replaced by a new method of engagement 
today. Bloxham (2010) examined the formative assessment role that continuous quality 
improvement (CQI) had on student perceptions of quality and concluded that students did 
like to be asked for their feedback and see changes incorporated into the course. Hill (2014) 
noted that graduate students and their level of satisfaction with courses are a good measure of 
quality as these learners have had years of experience not only as students but also, possibly, 
as teaching assistants. 

There are a number of diverse aspects of the design of an online course, as well as the specific 
features included in the course that contribute to its overall quality. The role of institutional 
support cannot be underestimated but was not included in this study. Institutions should, 
however, decide how they will measure quality in their online course offerings independently 
of traditionally delivered, or face-to-face classes (Mitchell, 2010). 

Study Design 

Between June and December 2015, research was undertaken to examine the notion of quality in 
online course design. Based on the work of previous researchers (Shelton, 2011; Roehrs, C., 
Wang, L., & Kendrick, D., 2013; Monroe, R. M., 2011) attributes were identified and potential 
survey questions were culled. Using this collection of attributes, new survey questions were 
designed that limited the instrument focus to these four domains: participant and course 
demographics; course design process and team; measuring and improving the course; and 
course content including instructor actions. This online survey instrument was used to 
question instructional designers and instructors (i.e., faculty) who specifically taught online in 
the United States. 

The survey link was sent by email, in the Fall to 2300 individuals who were members of the 
Association of Education Communications and Technology (AECT), 90% of whom lived in the 
United States. The study was limited to United States-based instructional designers and 
instructors. In the two weeks that the survey was open, 119 AECT members clicked the link 
agreeing to participate in the survey, with 103 continuing past the consent to the survey 
instrument. Of those 103 participants, 21 were excluded as they did not enter any data at all; 
two were excluded as they indicated they only taught in a face-to-face course; one was 
excluded as the participant indicated being a student, not an instructor or designer; and nine 
were excluded as they taught at institutions outside the United States. Although there were 
three participants who did not fully complete the survey, their responses were retained as they 
contained at least partial information. This left a total of 70 participants for an overall response 
rate of 3%. 

The Survey 

Information was collected to determine what role the participants played in the course design 
or delivery, and to what degree they were able to make changes in the online courses they 
built or taught. Participants were then asked to indicate on a Likert scale whether the design 
process or content features were 1 = always present; 2 = occasionally present; 3 = rarely 
present; 4 = never present in the course; or 5 = do not know). The design process questions 
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explored who was included on the team building the course, how often the course was 
reviewed and improved and if, and what type of, a course quality auditing measure was used. 
The survey instrument is included in Appendix 2, and the domains are presented in Tables Al, 
A2, A3 and A4. The University of British Columbia's Behavioral Research Ethics board 
approved this study. 

Participant and Course Demographics (Section 1) 

The first section of the survey asked about demographics. Of the respondents, 74.3% were 
female, 24.3% were male, and 1.4% preferred not to answer. The majority fell into the age 
range of 36-45, and the majority (41%) classified themselves as both instructor and designer of 
an online course. Participants primarily worked at 4-year public institutions in graduate 
programs taught entirely online; the median time they had been in their current role was 
between 4-6 years. There were, however, a significant number of participants who identified 
only as an instructor designer (35%) and those who indicated they had been working in their 
current role between 7 - 15+ years (38%). AECT is a professional organization whose members 
and activities are directed toward improving instruction through technology, so the 
assumption was that there would be awareness, if not a propensity, to support the best 
practices within higher education and online learning. The participants, overall, were an 
experienced group of educators in online teaching and learning. 


Table 1: Survey Participant Demographics 


Role 

Gender 

Age 

Length of 
Time in Role 
(Years) 

Type of Higher Education 
Organization 

Course Type 

Delivery Mode 






NC UG G 

Online Hybrid 


(1) Instructor 
(N=12) 

F=8 

M=4 

18-25=0 

26-35=2 

36-45=3 

46-55=3 

56-64=3 

65>=1 

< 1 = 1 

I- 3=2 

4-6=5 

7-10=1 

II- 15=3 

>15=0 

2 yr public=2 

2 yrfor-profit=7 

4 yr public=0 

4 yr private not-for-profit= 0 

4 year for profit= 

Other=2 

1 

3 

8 

12 

0 

(2) 

F=20 

18-25=0 

CO 

II 

V 

2 yr public=2 

2 

11 

12 

21 

4 

Instructional 

M=5 

26-35=9 

1-3=5 

2 yr for profit=0 






Designer 


36-45=9 

4-6=9 

4 yr public=16 






(N=25) 


46-55=5 

7-10=6 

4 yr private not for profit=5 








56-64=0 

11-15=0 

4 year for profit=1 








65>=2 

>15=2 

Other=1 






(3) Both 

F=24 

18-25=0 

< 1=3 

2 yr public=4 

4 

11 

18 

24 

9 

Instructor & 

M=8 

26-35=1 

1-3=10 

2 yr for profit=0 






Designer 

Prefer Not 

36-45=13 

4-6=5 

4 yr public=20 






(N=33) 

to 

46-55=9 

7-10=6 

4 yr private not for profit=1 







Answer=1 

56-64=8 

11-15=4 

4 year for profit=4 








65>=2 

>15=5 

Other=4 







NC: Non-credit course 
UG: Undergraduate course 
G: Graduate course 
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Data Analysis 

Data analysis was conducted using SPSS 22. The working hypothesis was that there was a 
difference in which quality attributes were incorporated into the course depending on 
participant roles (instructor, instructional designer or both instructor and designer) within the 
course design. An ANOVA test was performed for each of the three sections of the survey as 
outlined in the Appendix 2 (Table A2, A3, and A4). 

Course Design Team (Section 2) 

The second section of the survey asked about the course design team. The majority of 
respondents indicated the online course design teams always included the instructor (81.2%) 
and an instructional designer (68.1%). At the same time, 79.7% indicated a librarian was rarely 
or never present as part of the course design team, and 73.9% of respondents indicated a 
content expert, who was not always the instructor, was included in course design. Of those 
participants who categorized themselves as the instructor (solely - and not the instructor and 
designer), 50% indicated they included an instructional designer always or occasionally in the 
design of the courses they taught; all but one respondent in this group were teaching in a fully 
online course. 

Much of the recent literature has looked at the process of online course design and delivery as 
contributing to the quality. The method of design that uses collaboration between faculty and 
instructional designers was likely to result in the identification of a quality course (Bates, 2015; 
Brown, Eaton, Jacobsen, Roy, & Friesen, 2013; Chao, Saj, & Hamilton, 2010). The inclusion of a 
librarian was also key to ensuring quality by helping to identify teaching resources, to address 
copyright issues, (Bates, 2015) and as a guide in the alignment of research assignments with 
library resources (Mudd, Summey, & Upson, 2015). 

Measuring and Improving the Course (Section 3) 

Participants were asked if they were able to make changes in the online course; all indicated 
they were able to make changes: 47 (67.1%) said they could change anything they felt was 
necessary, 16 (22.9%) could change anything as long as it did not deviate from the syllabus; 
and 4 (5.7%) were able only to make changes due to technical issues or grammatical errors. 

With regards to measuring course quality, when asked about rubric use in the course, only 
56.5% said a rubric was used. The majority indicated they used a rubric designed internally by 
their organization (48.7%); close behind was the Quality Matters rubric (41%). The least used 
was the Quality Scorecard provided by the Online Learning Consortium (5.1%). It was 
encouraging to see that 68% of all participants indicated their course was reviewed and 
improved annually; however, it was notable to see the remaining one-third (32%) either only 
occasionally, rarely or never reviewed or improved their course on an annual basis. Of the 
respondents to this question 11.6% indicated they did not know how often the course was 
reviewed or improved. This question was explored since the use of rubrics as a measurement 
of quality features was widely supported in the literature (Newhouse et al., 2013; Parscal & 
Riemer, 2010; Pina & Bohn, 2014; Roehrs et al., 2013; Shelton, 2010). 

An ANOVA test was performed to determine if there was a difference between the groups 
based on participant role, on whether or not a rubric was used to measure the quality of the 
course and if the course was reviewed and improved annually. The t-test indicated that there 
was only a difference between groups on whether the course was reviewed and improved 
annually. 
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Table 2: ANOVA test - Course Reviewed or Improved Annually 




Sum of 

squares 

df 

Mean Square 

F Significance 

Course is 

Between groups 

22.623 

2 

11.311 

3.182 .048 

reviewed and 

Within groups 

234.595 

66 

3.554 


improved 

Total 

257.217 

68 



annually 







A comparison of the means shows that participants who were both the instructor and the 
designer of a course were most likely to review and improve the course annually; the least 
likely to do this were those who identified themselves only as the instructional designer. This 
may be as a result of the instructional designer being assigned a course during a design cycle 
but not having any responsibility for follow-up revisions after the design was complete, 
thereby leaving this to the instructor for the most part unless the design team (instructor and 
ID) was tasked with engaging with future iterations as part of the design process. 


Table 3: Comparison of Means by Role - Course Reviewed or Improved Annually 


Role 

Mean 

N 

Std Dev 

Std Error of Mean 

Variance 

Instructor 

2.33 

12 

1.775 

.512 

3.152 

Instructional Designer 

3.29 

24 

2.074 

.423 

4.303 

Both Instructor & Designer 

2.03 

33 

1.776 

.309 

3.155 

Total 

2.52 

69 

1.945 

.234 

.783 


Course Content Features (Section 4) 

When asked about the attributes determined by the literature to be quality features in an 
online course, there were clearly some of the attributes that had a high percentage of always 
being present in the courses and others that were rarely or never present. The attributes that 
participants specified, regardless of their role in the design, which were always in the course 
included: 

• Communication between the learner and instructor is specifically outlined at the start 
of the course, i.e., where to send specific requests or how long to expect before feedback 
from the instructor (86.8%) 

• Contact information for the Learning Management System (LMS) is included at the 
start of the course (80.9%) 

• Deadlines for completion of work are regularly scheduled throughout course (88.1%) 

• Instructions on how learners can meet course objectives are clearly stated (83.6%) 

These results are encouraging, as they seem to suggest a propensity among experienced 
instructors and designers to incorporate literature based quality features into their online 
course design and teaching process. 
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Other important quality attributes that had a more uneven uptake included: 

• The instructor did not always outline the expectations for discussion posts (32.4%). 

• The incorporation of synchronous opportunities in the course was distributed 
across the spectrum from always to never (and do not know), with the majority 
(23%) indicating that this attribute was only occasionally incorporated. Although 
this study was focused on asynchronous courses, a course may include an optional 
synchronous opportunity for the learner to connect with other learners or the 
instructor. These sessions may be recorded for review at a later time. 

• Having learners present their projects to peers was only occasionally, rarely or 
never present in the course (> 63%). 

• Student peer assessment was used in the course as formative assessment. Only 
20.9% said this was always in the course design. 

• Samples of excellent work or case studies were used to demonstrate expectations 
instructors had of the learners for an assignment. (Only 18% responded this was 
always present.) 

These results suggest there is inconsistent incorporation of quality features, especially those 
that involve constructivist strategies for active learning and engagement. If learners, as an 
example, are not presenting their projects to peers but only submitting them to instructors, 
they do not have the opportunity to learn from each other and practice critical assessment. 
Only 20.9% of participants indicated they included having learners assess each other's work in 
their course design, yet this is a consistently used method of formative assessment (Romeu 
Fontanillas, Romero Carbonell, & Guitert Catasus, 2016) that can increase student engagement 
(Weaver & Esposto, 2012). Synchronous opportunities allow the learner to connect with others 
in the course as well as the instructor, which may result in higher levels of student satisfaction, 
retention, and grades (Jaggars & Xu, 2016). Although, Jaggars and Xu (2016) concluded that 
students value interaction with the instructor more than student-student interactions, that may 
be dependent on the student demographics (community college versus other levels of higher 
education, such as graduate studies), a difference that may require further study. 

A one-way ANOVA test was performed with 'participant role' as the independent variable 
and the Course Content Features (Appendix 2, Table A4) as the dependent variables. For three 
of the course content features, the t -test results showed a P value of less than .05, so the null 
hypothesis was rejected. Results for these three content areas are listed in Table 4. 


Table 4: ANOVA: Course Content Feature Incorporation Analysis by Role 




Sum of 
Squares 

Df 

Mean 

Square 

F 

Significance 

Synchronous 

Between Groups 

8.059 

2 

4.029 

3.127 

.050 

Opportunities 

Within Groups 

83.750 

65 

1.288 




Total 

91.809 

67 




Learners Present 

Between Groups 

21.399 

2 

10.699 

9.697 

.000 

Projects 

Within Groups 

71.719 

65 

1.103 




Total 

93.118 

67 




Learners Create a 

Between Groups 

6.392 

2 

3.196 

4.986 

.010 

Product 

Within Groups 

41.667 

65 

.641 




Total 

48.059 

67 
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The means of the three groups were then compared as shown in Table 5. For these three 
quality course content attributes, the data show that the Instructional Designer was least likely 
to include the items, whereas, the Instructor was most likely to include them in the course they 
taught. 


Table 5: Comparison of Means by Role: Synchronous Opportunities, Learners Create a Product, 
and Learners Present Projects to Class 


Role 


Synchronous 

Opportunities 

Learners Create 

a Product 

Learners Present 
Projects to Class 

(1) Instructor 

Mean 

2.08 

1.17 

1.42 


N 

12 

12 

12 


St. Dev 

.900 

.389 

.669 


Variance 

.811 

.152 

.447 


Std. Error of 

Mean 

.260 

.112 

.193 

(2) Instructional Designer 

Mean 

2.83 

2.00 

2.92 


N 

24 

24 

24 


St. Dev 

1.239 

1.063 

1.248 


Variance 

1.536 

1.130 

1.558 


Std. Error of 

Mean 

.253 

.217 

.255 

(3) Both Instructor and 

Mean 

2.13 

1.50 

1.97 

Designer 

N 

32 

32 

32 


St. Dev 

1.129 

.672 

.999 


Variance 

1.274 

.452 

.999 


Std. Error of 

Mean 

.200 

.119 

.177 


Discussion 

The data in this study show that many attributes of quality learning are being incorporated 
into the course design and the course content. Some items are clearly being used: the design 
team always including the content expert and instructor; communication and navigation 
instructions outlined, deadlines for completed work regularly scheduled throughout the 
course, instructions on how learners can meet course objectives clearly stated, contact 
information provided for the Learning Management System and the library, although this 
attribute is included less often. Yet even with this level of quality, there remains much room 
for extending the number of features that a course design team might regularly include in their 
courses as outlined here: 
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• synchronous opportunities 

• learners present course projects to peers 

• learners produce or create a product at the end of the course 

• learners explore content beyond the boundaries of the course 

• samples of excellent work or case studies are used to demonstrate expectations the 
instructor has of the learners for an assignment. 

One of the more important additions to this list would be the recommendation to include a 
librarian in the course design team, an action likely to result in richer learning 
experiences. Next would be to recommend that courses be measured against a rubric for 
consistency in quality, and reviewed annually as part of the good practice of updating and 
improving the course. As online instructors become more knowledgeable and skilled with a 
broader variety of multimedia software or tools, we would expect to see more inclusion of 
these types of items as part of the course design. 

This study did produce paradoxical findings, as the instructional designers were the least 
likely to include some important features. As professionals in the design of instruction, it 
would be expected that they would know and practice best guidelines and insert opportunities 
for quality online learning. 

There is a critical need for 'good modeling' for future instructors, within the quality design 
process. New instructors, often graduate students learning their craft, need to see quality and 
understand its value, as they may, one day, become a more formal part of the academic teacher 
world. 

In addition, college students, and especially graduate students, who are adult learners, require 
specific course features to enhance learning. Samples of assignment expectations provide 
scaffolding to adult learners, and case studies provide relevance of knowledge to their 
personal lives (Knowles, 2005). Beyond this, it can be argued that the adult learner could 
benefit from social interaction for learning often provided within online discussions and 
synchronous opportunities (Cercone, 2008). 

Some courses will not need all these attributes. As Mitchell (2010) emphasizes, the institution 
needs to initiate a discussion about how they will assess quality in their own online course 
offerings, and within other auxiliary support such as student services. This is all determined 
by many factors including accreditation requirements, stakeholder expectations, and current 
ideals, as well as the needs of the faculty teaching and the students learning. 

Conclusion 

To show quality in online courses, one must include the design process, specific course 
features, and student services, as well as instructor delivery. The set of quality attributes 
selected for this study, for the most part, support social constructivist strategies (Bandura, 

1986; Fosnot, 1996; Vygotsky, 1926) to encourage active learning and student engagement with 
the course content. The literature concludes that these strategies are the most effective in 
increasing student satisfaction and student learning outcomes. The first question was 
answered and the survey instrument was built using a selection of these attributes. The study 
limited its exploration to four areas: the course design team, measuring and improving the 
course, course content features, and instructor actions. 

This study showed a high percentage of respondents — who are experienced designers or 
instructors of online education based on the number of years working in their roles — are 
incorporating quality features in their courses. The results also indicate that there are still a 
number of those working in online course design, who only rarely or never include many of 
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the features. Comparing between the roles, there were three areas where the instructor of the 
online course was most likely to incorporate these quality attributes. This was a surprising 
result given the professional nature of the instructional designers involved in many of the 
designs. 

Institutional support was also a critical element to ensuring quality attributes were embedded 
into the online course design process. While this might primarily be in the form of faculty 
development and training, it was also an important part of the support of practice, where 
courses are expected to be reviewed and improved annually. There are many questions that 
this study elicited that a future study might explore: 

• Why did course designers NOT incorporate these quality features? 

o Could it be they just do they not believe in them? 

• Might the structure of their institution not be supportive to quality considerations? 

o Might this be creating this lack of incorporation? 

• Is making an online course using quality attributes just "too much work"? 

o Is this the way they have always worked and don't want (or cannot see the 
need) to change? 

• Is time the issue? 

There may be any number of reasons underlying the decision to include or exclude quality 
attributes in online course design that need to be further explored which would allow these 
barriers, either real or perceived, to be removed. In the case of instructional designers there 
may be constraints on the latitude they have in incorporating innovative technologies. 

Although all participants indicated they could make changes in the courses they designed or 
taught, the extent to which the instructional designers can influence the incorporation of 
course features is an area that is recommended for further exploration. In the case of 
instructors not incorporating more quality features, a qualitative exploration, specifically of 
instructors of online courses in higher education could shed more light on the reasons for this. 
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